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(54) VOLUME RENDERING PIPELINE 




(57)Abstract: 

PROBLEM TO BE SOLVED: To realize a 
rendering pipeline for volume rendering allowing 
user's selection of processing order in stages of 
the pipeline. 

SOLUTION: This volume rendering pipeline 
includes the plural processing stages such as 
gradient estimation 220, interpolation 240, 
classification 230, lighting 260 and composition 
270. A first multiplexer 25 1 connects an output 
part of the first stage 240 to an input part of the 
second stage 230. A second multiplexer 252 
connects an output part of the second stage 230 to 
an input part of the first stage 240. An input part 
of a third inultiplexer 253 is connected to the 
output part of the fu-st stage 240 and the output 
part of the second stage 230. The multiplexers 
251, 252, 253 respond to a selection signal and 
set the rendering pipeline to process volume data 
set. 
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(54) B!MO«»] 4fJa.-A^>^'U>^'>'W:7'7'f> 



(57) iwm miE^) 

\.Wmm i^-^) :^-J^Vy^'^)y^nMi'vAy 
\t. ^E*t^2 2 0 . ffl^2 4 0 . ^^:«-(t2 3 0 . M 
Bfl2 6 0Sl/f-^Bg2 7 0^^01gic^7)^)lax-r-i^'&^^ 
•Ca^S, ^1 v;W^r^;7-9-2 5 1(±, W.\X'f~~J2 
4 0«0ffi^l^S:m2^T-*s^'2 3 OcoA^^lSCtC^iglLT 
v->^. ®2vyl^^ri/^-if 2 5 m2Xf— ^-'2 3 
0 c7)fij:i3gB 1 Xt^— >' 2 4 0 i?) A::b gPt;:fiM L T v 
-i>. ||3-v'yl'f-rU'^'-9-2 5 3{±. A:bgB§:^ 1 >^■r- 
v^ 2 4 0 c73ai:'3gB t ^ 2 Xx- i-' 2 3 0 <?)aJ:'ja5tCtgSg 
LTjsD, ■7;l^f-n/^7-9-2 5 1 . 2 5 2, 2 5 3{i. 
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IcSf^LTV^I,^ 2-7;Pf-rw^-9-fc , 

L^cm 3 v;Pf-7V ^ -9- 1 1 X. , 
±fBmi, m2Sr/^3«Ov;L'^rP-^'-9-3t)^\ ±iE^l 

y. 

*iB2 tclBiJi^-Ky A-Ai/y^'J >i^vsM r^^ y, 

'J a -A 1/ y y 'J >'^VnM ^ 
C 6 ] ±iBII 1 . IS 2 Sl/^'lg 3c7)-r;l-f-ri/^ 

^ t ^ irnst -r ■& if *Ji 1 {cieifeco^if u A- A 1/ y ^ «j 

IB^o^'y A-A u-y:^ u yyjM y'y^y, 

mm.-ti> z t i:mit-t6msm i izimff)^^v ^- 
Auyyv yy^u ry^y. 

U A-AWy^'U >'^V^Mr7^ y. 

imim. 10] #7 -yi/ KJi, ±Mi-!r^9 -tMz , m 

[fii^Jill] !^^c07-f-yH<{i, ;^<•Ui-A;^rf• 



i-A u- y u y 7-7 y , 
[|f^Jai2] #7^-;UF{i, #^S'5r'?Wjit/:: 

tcKKco;KiJjt-A^yy'jy^>NM7°7'f y. 
[|«*« 1 3 ] ±fe^ieft^xr-i^'<i. 1S^<^7 >f 

i t ^!t*m t -ri.ii«« 2 tiE«co-if y j.- a y ^ u 
y^ysMr^-f y. 

[0 00 1] 

J^(DUyy^jy^l,zmt. mzj-^^y'y-i ycoxx-i^' 
l^za (t S i!iSJi»*^'i— J: -o T SiR^fgt o X V > 

^ i^y^'j y^'^'M r7-f yt^M-rs o 

[0002] 

{vm<^mn TKUi-Au-yruyj/iois^iifj 
Au-yi/'j y^'ji, 3<?:7tf-:5'*^'am$ix.l.'i:>^*^' 
h^-oy\^ a.-if ^yy ^ y9xa)mmzUiUmm 
^flh, -i<UA-Af-^{±, #(*:^m!H!J^Ej^«^ft 

0, ^^X\i. r—^inmcfh^tX'f-'^^zX'y 

xm^tihm-^Li-zm^^m&immL'^-t<-thhcn 
x$ih, 

[0003] -•K'Ji-Aixyyyy^tioT. 
I,. iiS, 4<'JjL-A^y^>jyi?'T'fl6ffl 

'^^'r!v<Dm^<n>3<xytwif^m-T—?mBX'hh . 

#;K^'-fe;L'C7)J^S (x. y. z) {±, 

[0004] ^s^-^Mi^ nm^x\t^'T)v<nmhm'^ 
<m^m.^mLx\^h. ^i.tje*sffit<^s.-j<u^-A 

hzhifx^h. vy-fuyy^^iz^ :^^'p'^)V<r>{mi. 
^htzisb^z 2iJ:7cli«M±tS^.$ixl, ^ k \.zts:h^h 

T-wm ( RG B « ) (nmzm.mmzW'ox^m^ix 

[0005] l^y^'J yy^\,zUIUim^^fi.h—-:> 

<D^m±. vA^^x^yy ^^wm) xhh. -m. 
(r>imm.ti<i¥^<^-^)\^<r)r\^-imLxi9dt^ti^,. % 

%m.z^'^fzm^<r>^\.zn\.xm 
mb^yy'j y^^ti. ttz^yr^j y-i^^fitzmi:m 
%mz^m-th^^-^£^Sitii'^t>tix\^h . ixh 0 1 . 

--K^'-tyi/fflJi, il«RGBa<7)--i<^'-fe;Hc^^§ix. -e 
ti^*^ <^:t3KI8tfp^oT^'53iL-9-y7-u y^^'^tii. 
-ymmmzmm^tihijCDifhh . ^^-rtico^^h. 
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[00 06] uyiTVyy^UTy^ y^nu a.-J>,uy 

ti^Xx-i'VNMT^^ LTltf£$ti.TfcO. 1 9 
9 8^1 Ifl 1 2Bt=>^r5-ft^t:J;oTajP§tL/i 
*ll#fflFaiffilSO 9/190, 643 r||Bfra^Orf-r 

[ 0 0 0 7 ] H 1 (i, if^9^}\/m-if^9 U 1 0 

H^ian^iiTv^s^t^f r5 y 1 0 0 LT 
-g., ;lf^'-b;Hl(±, %~fxv^7.'t\.X)M-Ty-iy(r> 
if-9^)V)^-jyr 1 1 Of^{cgg;^aiiiS. imr)-L^ 

z.fii^(r>m:'S.'^-thm^xn~~v\ 2 otc^sni.. 
-£jE<7)x^^at/fyBg:^s±. xr-v-'isotfc 

-i^'l 4 0fc^§il. 4 5tc 

1 0 9t2iB1S§ii/sS^<?D3t46<?)M 

[0008] 

y S:|ilB#tc p yf y^'-ft i t \f¥^mX'h 
[00 09] 

[iSM5:»^-r-l./ti6co#S] *5|BBi:, S^fiO^taax 
X- W L;t --n" U - A I- y i?' 'J y ^'y T 7 V S: 

9^\zi.->x'K^^\,z^m.^fix\^h. 'm\'^)V'^yv9 

Xhmz^mX^. t.fzW.1'7}V^TV9^\t. i^^Xr 
-^-'^^Ai^aSS: i 0-'-?m<^XT-iy<Dliif}mzimL 



^■yht:^m-fhtzi^f,z\^yy')y'fjuryAyi:m. 

[0010] *%HJ(r)— OiO^fflT'ti, j.-Af- 
[00 11] 

[^HBonMojgffi] 112 {i, *»Bjt3^s^^prtg^ 
y U y y 2 0 0 CO h y 7°K;i-<7)7'a 
'y^^mm^^Lx^'^h. juyyAy2oo\t. ^^^-^ 
U 2 0 1 LX'^y:r)VX\t^^i^'^)V^ 
WB-L.. U 2 0 9 ttU^j t tTBStrlEItt 

-S. ;f:^'-fe;i^X{i-^yr;Wi. |simi;f:ua-A>c5-f 
tT'1^^'-k;U>'N'-y 7r 2 1 OtK^^iiiixS. -^ffi 
Ji, XT->''2 2 0(Ci3V^Txy z<7)^:}i-$r^*^S^< 

m^^iifz^yrMzm-^\>-^x\m'&^ixt{: 

[00 12] --f<:;'-b/P2 2 Hi, ||2;^r-i''2 3 Of 
i^'2 4 0ti3i^Tff*)ix&„ ^iE(7)fiSi:ffiS«i, -<J^ 

*^Bj<7)S!;st LT, mmzX':>x:!r^'y'^jvtmni^ 
^t{:xn'-i^[zii'ox^m^ixh)mti^m>(:>tih, 2 

X\^h. 

[0013] xdf ^-^-{i, 

yrvy^-^tifzT-^^m^t^. mmtfz^^i^-^)i' 

^'&.±xhfzMz^ mm^nmm.ii. 2ocoffl^o^^ 
"^^w^-t^^x'hh . 

[ 0 0 1 4 ] ifip^, ^yy) yy^tifzf—S'ii. Mx. 

\mttm^t i}^-^mmmcnimf<z^»)mi- (^t 

-^i3^H-t ^tih t . 2':>cr,m l,zmi\^ixfzmi<r>m 
X-mmiltf^X. -?■^^I^^t{i#^£L^V^m3^0fi^ 
i:%^Ztizy'j:h<^liym'M^mX'h^o. Ztl^cDm 

^ff^^^-^ji-^mm^ifL. mz^^miz^ti-^ 
^nmi-hm^xK)mn.x\,^i,. ztm. mmttz^m 
co^^ T<7)^ttc7)^^ fc V ^ 0 n(^m(>zim ^ tf 

[ooi5]fi!-pT, fUry^y2oo\±. mixy- 
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(C, '?)V^rV-^^25 2},i. |g2Xf— i^'2 BOeoai 
-n^-^^lXy—i^2AQ<7^X-nmzmW,LX\^h. V 
)V^-TV9^25 3\i^ |?2X7'->''2 3 0X{±miX 
•r- 2 4 0 cOl ^iNxm^ ^> (TDtH^J SrStR-r -S , 
[0016] Xx-i^'<?DS^fi|c7))i#{i, 3-)<^'7)V^-T 
^^'-9-2 5 1~2 5 3^<?)StRfl-^ (GCI/GI C) 
2 0 2{c:??&$ii?.. v^-fixcoii^i,. fflra^iut^ii 
trnW^^ixtz^yrMi. Wmx'r-'J2 6 Otz^$ii 
•?). Wf^-^iXfzRGBa<r>mi. X^-'J21 OX'^'^ 
*/::ai^JiB«2 7 Ui, li^;^^ 'J2 0 9(2ie'tt 

[ 0 0 1 7 1 ^nm rv^ y2o ox'li. mm^x^- 

>''2 2 0coai:tl{±. •JUiB03(7)^tJi-t^oT<<^l., Ztl 

3— :)S"|co-7;l'f-7Vi7-9-Sr/r-Lt:B3HJXx-x2 6 0 
[0 0 18] 3OC0V;l^f-ri'^'-9-2 5 1~2 5 3 5:3i 

mzm^-thztx\ sfa<7)Hgt=^M^Ht*. xji^ii 

^^(ttfOMt^MS:, XtiWalcOAS:. Xl,:i.^^l,fcr>7f. 

vtrxmmizmBM-t^. 

[00 19] GI C-W. fflra, ^^(tH3{i, G 
CcO^-KJi, fl#2 0 2(Cj;oTjl#^$ix^« GIG 

[00 2 0] ^y:?'U>'j^<i, -ffitc, fiP*>3 2x3 2 

(omi^)i-y°x'<mmimizmts, tmcotmi^mi 

^X'5o;J<:7-fe;Ws;-y7r 2 1 Oi,zm,^ThtrLh. #A 

yT>j>y<nWi^^MLm&i,zj:'ox^h&, 'jmt^K 

^ytif'-tJl'Com^Wiit. 3 7x3 7c7);K^-fe/UT'$>i., 
[ 0 0 2 1 ] -tJU^ -t^i? -fe^L-y s; y 7 r 2 1 0 (CSg^^ 
2=tfWi. m&Xy-tx<7)-^^m^m^^^^Ztl,z^j: 
^-ox^ \,^-fti(Dm&ft>zi,^ -n(r)Xy-( xtfim 

m%'mm^x^-~j2 2ot^^9-^)mmx^- 
2 4 0 cc«*&-r s <^^zm-mit ^kx^^^. 

[00 2 2] ll3tCfc<,^T. s + 1 i: s + 2c7)7<;K7) 

flii:^X5-^Xs, s-lSt>'s-2{i, T^^-tJi^i^im 
Xi^-i^iZimtX^^h. Xy^Xs-3l,±. m<DXy 



X'^l>zmm-^fiX\-^tii>(OX'$)'0 . 31^E2£^^'5r-5•cv^ 
^7^;^s-2cr)#^*^'^T-rSt, X5-^>?.s- 
2t^i:LTV-i?$n-g.. mz. Xy-^Xs-2. s 
-3*<;Jfi?-b;l.M^a^i7)ffLv^tgSifitc^'9. flfi:^3!ai 
m't=Jj.~f\^ii^Xy-i Xs + l^ s, s-lA^(i>-?-ix^ 

[00231 W«S>^f—i^'22 0{i, Xy-fXs- 

1 tcfc(t'5.*:i5'-b;k'St*3(tl.^3E5rX7-< x s , s - 
2(:fcttl.*'^-tr;Ml3&-^>it^-rS, Ztl(><r)^mii. 
mz:3.=.-y h22 8l,zi:^mm(r)fzlsf^l,z^ 2ocOX5-f 
XcOiUi2M'-y7T2 2 5tiS'lt$iX^. X7-<;^s-2 

t^E^ N' -y 7 r icieig $ ixT V %S . 
[00 24] <K^-fe;l.cr)fflra24 0tWfflra2 

2 8**|5lB#{Cjiff-r'5>« BP^s ^yrfUCOXy-^ X'fyi^ 
ifs^^}V<r)Xy^Xs-\^ s-2ti'h<^i^"9'z)V<§,^ 

Ura-tSCitTIf ^ns, ■9-y7-;Pc7)iifflc7)X7-^xtc 

t3(tS-^>'7'Uyj/;StCfc{tl.^ie«±. W^^'■/7r^C 

v ''T^IB:^ 7 ;^ s - 1 1 ^E;^ 7 X s - 2 o^E 
^«PB^-ri> ^ i:T1#^>iil., ^y-TiVhW^^Mzmi 
CO— oiO^i'-^T.^-f 7.*i, mtM<7)7i<^-^)lt^W.a) 
Xy^ Xt^i^'i^i^ixh^ 

[0025] mm^ritzt'9'^)V(nm,±. mzmm.'it^'f 
x'7—'J23 0{,zmm^ti. i<^mm-^nx'r-i>23 

0{i, ^Kt:,<r>m.t:m^^tttzRG'&a(/)^mz^mt 
RGBafflfc^Eti, <5:t;H3BHX7^-i;^2 6 0tC« 
m.m^z-^WLX^--i^27 OiiZ^l^^ixh. 
[0026] Jrri/a :x^:-K-C{±, Vi-rn*^cO;J1"^-fe 
)Vcr>y -f ->'WH*>^>^ESrtt^-r.?.iOlc^EiiS:^7'- 
i-'2 2 0Srffiffl-fl.ftt)Ot=« ^UEJi, fi?iJ;t«f3Oi0;K 

[ 0 0 2 7 ] GCI --^E. ^i^^tt, ttPal 
Ill4«i. K{Cfi-^2 0 2tcto-CMtR§ix-g.GCI^:- 

vm(r)\yyy^)yyT)V-d^)XA^mT^i.x\-^h. ^v.^^ 
■fe;WN'-y 7 T 2 1 0{±. ±M<^i. -5 tc^SSn. i/c^ 

Eftsro-bxtRtT-S)!., ffia^ji, 

ti. *:^-fe;PX^-fx>'s-/7r*^^, m*>xy-^xs- 

ii}^^imi.^ti. mmi>zmmwrxr—=J2 3oi>zx-^ 

XRGBami^Z^m^tlh. Zili^li. mz2'Z>(r>Xy 
-f>?.cORGBaVN''y7r 2 3 5CtE'ti$ixS. RGBa 
^'N y7r 2 3 5{i;. 20(7)X5'1'7.c7)^E>''N-y7T 2 2 

[0028] 2^<r>XyA Xtf^, J^:tRGB aOMUX 
x->''2 4 OtcA^J^iX. ^^T'liPa^iX^cRGBafi 
{i. nWnW^hWi^'LX^'^Kh . Zfit:>\i. Mz 

w^^fifz'mDim.z^ w^x^-i^t^^x^-i/h 
(cjiffl^iiT. ^^(m%.<m%^-h2i 1 sr^-r 

I.. zixh<r>Wik(r)2^o:i^V^ 3.-Mt. (12~3c7)t 

|s]fc t'$)l>. 
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[00291 -OC0#JaS<)^;trv 3 y^- K t nrtST- 

•fe-y hi:U>-:ry>-i/t-h'tC*tLT{i:, Mtimm^ 
coo 30] -7;l^f-7Vi?-if2 5 1— 2 5 3{i, ffilcOX 

[003 1] yV^i^~f)i't{:ri^9-^)Vyit—'7y\- 

OOJi. liSctoy -yPH ( Vi. ■ - • . vj 5 0 1 
— 50 9€:^"LTV^-?.<, 7 5 0 1 — 5 0 9c7)# 
*::?-fe;i.5 0 0lcfcv^ffiti:gi:LT#S$ti. 

[ 0 0 3 2 ] 7 >f -;UH(±, ^^tlt3<'Kjt<r>nMmX 
WAcr)m<r)m^. ^<7)m. 7 ^ 5 0 1 ~5 0 9 

{i. CT. MR I. PET. SPECTS.t>'@^ 

mx^-^y:f}-i^mm^ti6^mmmi^-r^hff)X'h 

T 5':?«p:5r-,^^^{cj;|,7^-^c7)3SJgfi$ieiiT'# 
c7) 7 ^' H (c^-^ L T V 1) ^ < c7)SiJ ^ O-K U i - A i: 
[0033] ^ -if »J a-AA^ 0iJ 

U >'^$ixl.SiM**^fl6«0gP^^^|5l^-r.|. SCOT'S) 
\-^-rti<r)^^ij z>.-M.T-:$'-^zyh<r)rcMzh. y^-}V 
[0034] a-^cO^BfltC^i.vyl/f — 7 ^ -yPHc7),+' 



^'■feyps 0 otcjiftLT. i-if{i. ^ies:^tit'c07 -f 
S^a. ^(nmt'k^^^hf^MzKff^if^^^jVy ^-)V 

[0035] 7 -f -/PKc?)gmc03tJ6t'. a 

V\ Stc, ^:?-b;H^c7)^7^'-7L'F{i, ^Jx{f^7 

i^TgiHB$iiTV^^ J; -5 ^7^dri^7';^<c#:^'-fey^ 
{±, -mts:1isi^X'if^9'z)Vyt-'^-jV(r>^X<mJA'r 
-X$rffia!'i46t*a£7)7V-AV-^'S:ffiffl^tgi: L 

[0036] y ^-)VYyt—'^-jV<r>\^i>x-^ 

ffl^tgtci--?. 7 ^ K7 -y h<?)^i;^X;5' 6 0 0 

^^LTvi^. —■m^mx'M. ^r^^-tjucoy 

«i, 7^-;l'F7*-v-y hlx^-'x^'iOiKHBtcioTS 
m^tih. Ztil,±. 7^-;PH (4-b'-y hco-7';l' 
CO) eiOCOVAXt, -?-cO;j<^-te/I/6 2 0F*JgPt^t3tt 
S7>f-;l'h'<7){>iat . B« (Mtc, 4-b'>yhO-7' 
7-f-;Uh'*J'?-iOSIl$it/i<slfflO:Aci6(C# 

^tcT) ($IJP63 0) t2r^«-r-i>8-t--y h (7 : 
0) cofB^l^t^o-CV^I,. 

[ 0 0 3 7 ] 3 y h o-;Ub'<y F {4. 7 h'*<a-5 

xffK^u ryA y<7^'f~r:m\,zm^-thfzMzy ^ 
-)v\^t'^}i<r>i:o\,zmm^Khi}^k^mLx^^h . y ^ 
-)V¥\±. ^m.<^imkmrM-t^tx\ x^immm 
^x«4«ffi*%i)-^cov^m*^*-^> t'-y h ^nm-h 

[00 38] nvho— ;l'=0 : ^cO,-^'^^r;^c07 -f 

KB [0.. 1] [zanhm^mLx^^h^ 
^.L<^m^mL'rtmbLxwi.htih. T-rMmzm 
^-ThtzMzmmL^mwrn Ltz o . ximA^-th 
tzi^iz^ m^'ML»m<7)imi}K mmzmm^tix. 

ayhn~/U=l : ;l-:^-fc;l^c07-f-;l^K«4. liH 
[ - 1 . . . + 1 ] liZmfm^m m^M L-^mt LX 

3yho— ;U=2 : ±cO;K^'-fe;K07 ^• -;UH{4. r 

-^'assicjS^-ri.f^a^ic-ecoeffi^h'-y h < ls 

•y hti. «l?$fLl.« 

3yho-;P=3 : :tcO;K:?-fe;l^c07-f-;l^H{i, r 
-:5'ilS§{C®-^-r-l./^4^(C-e<0««^b'-y h (MS 
B) •C-S^$tli.*\ Xtm^ikXi^ixh. ftffi*^b' 
•y F{4. ffiff^ixl.. 
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[ 0 0 3 9 ] 0 jS ttM-^£7)S[^ 

[0 040] R^23ifficO|!ttCfc{t-&t'-y hi^t t 

I., ^flX\ F = V/ (2R - 1 ) (i, lEH [0.. 
1] tfc{tl.*a^t«^-CV^|,. BP*>, V«i0<?5B#, 
FttOT'S)"?. t/^V«<^-<7)ft;^nrtgffi(7) (2'< - 1) 

*>!.. OpenGL (^^P^Bi/'J-y^y H) Bgjffl^ 

[004 1] CLCL-ClKBB^ilfc^&a^O^Kl^^* 

mzmmthtmo^tifi. o. vvvvw. . . 

[004 2] fsDjRLc^^iSCti, RJ;0^<cOb''y 

m^tltzXir-fl-mi. (2R - 1 ) ?:^aLTV>S(7) 
T\ R{±, SCO ir^DjRL»Sj T-ft^. iixT', F*-' 

mmio.. i'\m\-(r>m^mxi,j:oizLx\,^6, z 

< R - b' v h cOiSlStc^ 0 ii-r 2 jiffifl kt'i^mfS.^tL 

[ 0 0 4 3 ] ^ 0 51 L«Sc^SlE^#^/^"^l!lOft»^ 
5fe-f#^cO^'93lL^^i!t<7)^^J6(cF2:?i»L. a# 

1 ) mHx^m<om'OMLm&Ri:i^iib^zkX'mm 

\,^Xcoimcr)fzMz:^^LX\^^. mx.i£. RA-f>2R 
lC^'93IL»SS:2fgt-|.{C«s #(CV+ (V«R) 

[0044] 7r^9-^z)Uy ir 
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1 Title of Invention 

Volume Rendering Pipeline 

2 Claiais 
We claim: 

1. A volume rendering pipeline including a plurality of stages for 
rendering a volume data set, comprising: 

a first multiplexer connecting an output of a first stage to an 
input of a second stage; 

a second multiplexer connecting an output of the second stage 
to an input of the first stage; 

a third multiplexer having inputs connected to the output of 
the first stage and the output of the second stage, the first, second, 
and third multiplexers responsive to a select signal for selecting an 
order of processing of the volume data set through the first and 
second stages. 

2. The pipeline of claim 1 wherein the plurality of stages includes a 
gradient estimation stage, interpolation stage, classification stage, 
illumination stage, and compositing stage 

3. The pipeline of claim 2 wherein the select signal connects the 
interpolation stage before the classification stage. 

4. The pipeline of claim 2 wherein the select signal connects the 
classification stage before the interpolation stage. 
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5. The pipeline of claim 1 wherein a gradient interpolator using a 
gradient buffer operates in parallel with a RGB a. interpolator using a 
RGBcr buffer. 

6. The pipeline of claim 1 wherein the first, second, and third 
multiplexers bypass a particular stage of the pipeline. 

7. The pipeline of claim 1 further comprising: 

a memory, connected to a first stage and last stage of the 
pipeline, ihe memory storing the volume data set before and after 
processing by the pipeline. 

8. The pipeline of claim 1 wherein the memory stores a rendered 
image, 

9. The pipeline of claim 1 wherein the volume data set includes a 
plurality of voxels, each voxel including plurality of fields. 

10. The pipeline of claim 9 wherein each field has an associated 
offset and a width in the voxel, 

11. The pipeline of claim 9 wherein a particular field stores a volume 
category. 

12. The pipeline of claim 9 wherein each field is interpolated 
according to different associated interpolation function. 

13. The pipeline of claim 2 wherein the gradient estimation 
stage extracts gradient components from a particular voxel 
having a plurality of fields. 
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3 Detailed Description ot Invention 
HELD OF THE INVENTION 

This invention relates generally to volume rendering, and more 
particularly, to a rendering pipeline wherein the order of processing 
in the pipeline stages can be user selectable. 

BACKGROUND OF THE INVENTION 

Introduction to Volume Rendering 

Volume rendering is often used in computer graphics applications 
where three-dimensional data need to be visualized The volume data 
can be scans of physical or medical objects, or atmospheric, 
geophysical, or other scientific models where visualization of the data 
facilitates an understanding of the underlying real-world structures 
represented by the data. 

With volume rendering, the internal structure, as well as the external 
surface features of physical objects and models are visualized. Voxels 
are usually the fundamental data items used in volume rendering. A 
voxel is a data item that represents a particular three-dimensional 
portion of the object or model The coordinates {x, y, z) of each voxel 
map the voxels to positions within the represented object or model. 

A voxel refM-esenls some particular intensity value of the object or 
model For a given prior art volume, intensity values can be a specific 
one of a number of different parameters, such as, density, tissue type, 
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elasticity, or velocity. During rendering, the voxel values are 
converted to color and opacity (RGB a ) values, according to the 
intensity values, which can be projected onto a two-dimensional 
image plane for viewing. 

One frequently used technique during rendering is ray-casting. A set 
of imaginary rays are cast through the array of voxels. The rays 
originate from a viewer's eye or from an image plane. The voxel 
values are re-sampled to points along the rays, and various techniques 
are known to convert the sampled values to pixel values. 
Alternatively, voxel values may be converted directly to RGB O. 
voxels, which are then re-sarapled along rays and accumulated to 
pixel values. In either case, processing of the volume data may 
proceed back-to-front, or froni-to-back. 

Rendering Pipeline 

Volume rendering can be done by software or hardware. In one 
hardware implementation, the hardware is arranged as a multi-stage 
pipeline, see U.S. Patent Application 09/190,643 "Fast Storage and 
Retrieval of Intermediate Values in a Real-Time Volume Rendering 
System," filed by Kappler et al. on Nov, 12, 1998. 

Figure 1 illustrates a pipeline 100 wherein voxel values are stored in 
a voxel memory 101. The voxel values are first read into a voxel 
buffer 110 of the pipeline as slices. The z-components of the 
gradients are estimated in stage 115 by taking central differences 
between voxels of different slices. Then, both the voxel values and 
the 2-gradients are passed to an interpolation stage 120 that calculates 
these values at sample points akDng rays. Next, the x- and y- 
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components of the gradients are calculated from the interpolated 
sample values in stage 130. These, along with the sample values and 
the interpolated 2-gradients are then passed to a classification stage 
140, and then a shading stage 145, where an illumination process is 
applied to produce the RGB or values representing the illuminated 
samples. Finally, the illuminated samples are combined along rays in 
an compositing stage 150 to produce pixel values for the base plane 
stored in a pixel memory 109. 

That pipeline structure suffers because the ord^ of processing data is 
fixed by the arrangement of the various stages. Also, voxel values are 
interpolated so that only interpolated samples can be classified. It is 
not possible to concurrently render multiple volumes acquired from 
different scanning modalities. In addition, the format of the voxel 
data is fixed. Gradient fields are obtained from the fixed format voxel 
data. It is desired to improve on this prior art pipeline. 

SUMMARY OF THE INVENTION 

The invention provides a volume rendering pipeline including a 
plurality of processing stages. The stages can include a gradient 
estimation stage, an interpolation stage, a classification stage, an 
illumination stage, and a compositing stage. The stages are connected 
to each other by multiplexers. 

A first multiplexer connects an output of a particular stage to an input 
of another stage, and a second multiplexer connects an input of a 
particular stage to an output of another stage. The multiplexers 
selectively connect the stages of the pipeline in a predetermined order 
to configure the rendering pipeline for processing a volume data set. 

In one aspect of the invention, voxels of the volume data set are 
interpolated before classified, and in another aspect, the voxel are 
interpolated before they are classified. 
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DETAILED DESCRIPTION OF PREFERRED 
EMBODIMENTS 

Figure 2 shows a top-level block diagram of a configurable rendering 
pipeline 200 according to the invention. The pipeline 200 takes 
samples or voxels as input from a voxel memory 201, and stores 
pixels as output in a pixel memory 209. The voxels or samples are 
read into a voxel buffer 210 a volume slice at the time. Gradients are 
estimated for x>/z-components in stage 220. Here, in contrast with the 
prior art, all gradient components arc estimated on voxel or sample 
values, not on interpolated samples. 

The voxels 221 are classified in stage 230. Interpolation of voxels 
and gradients takes place in stage 240, Gradient estimation and 
interpolation are linear operations and therefore can be interchanged. 
As an advantage of the invention, the application can determine the 
order in which the voxels and gradients are processed by the various 
stages. The reason for two different processing orders is a &>llows. 

Scanners acquire sampled data by making physical measurements 
which typically represent an integral over a small region of space. 
Adjacent voxels often represent the integrals of overlapping regions. 
In these cases, it is more accurate to interpolate voxels first, then 
classify the resuh. In particular, a voxel at a boundary of, say, two 
different tissues is likely to represent an average of the physical 
properties of each tissue. When interpolating, all that is done is 
moving the points of average. To make this useful, the classificatk)n 
function should present a continuous transition from between the 
colors and opacities assigned to the two tissue types. 
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Sometimes, sampled data may be pre-segmented (manually or 
automatically) into distinct materials, e.g., bone, muscle, cartilage, 
etc. Given such data, it would be inappropriate to interpolate between 
two differently identified tissues to obtain a third tissue that is not 
really there. In these cases, it is better to classify the voxels first, then 
interpolate the resulting colors. This is not so much an average of the 
physical properties of adjacent tissue types, but rather a blend of the 
colors at their boundaries. 

Therefore, the pipeline 200 includes a multiplexer 251 connecting the 
output from stage 240 to the input of stage 230. Similarly, a 
multiplexer 252 connects the output of stage 230 to the input of stage 
240. Multiplexer 253 selects the output from either stage 230 or stage 
240. 

The order of connection of the stages depends on a selection signal 
(GCI/GIC) 202 to the three multiplexers 251-253. In either case, 
interpolated gradients and interpolated samples are passed to the 
illumination stage 260. Illuminated RGB a values are composited in 
stage 270, and output pixels 271 arc stored in the pixel memory 209. 

In the pipeline 200, the output of the gradient estimate stage 220 is 
the three components of gradients. These arc passed to both the 
classification stage and the interpolation stage through the set of 
multiplexers. The output of each of these stages is each passed to the 
input of the other stage and also to the illumination stage 260 through 
another multiplexer. 
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By appropriately selecting the three muhiplexers 251-253, it is 
possible to do classification before interpolation, or interpolation 
before classification, or only interpolation, or only classification. 
These different modes of operation is now described in greater detail 

GIC — Gradient, Interpolation, Classification 

Figure 3 illustrates the rendering process for the GIC mode. This 
mode can be selected by signal 202. In the GIC mode, gradients are 
first estimated, then interpolation takes place on gradients and voxels 
in parallel, followed by classification of the voxels. 

Rendering proceeds a section at a lime, that is, in groups of 32x32 
rays. In order to process a section of rays, slices of voxels are read 
into the voxel buffer 210 two slices at the time. Each slice has only 
the voxels needed for that section. The number of voxels per slke 
depends upon the view direction and resampling frequency. The 
maximum number of voxels needed is 37x37 voxels. 

Reading voxels into the voxel buffer 210 can consumes the fiill 
bandwidth of the destination slices. Therefore, at any given time, one 
pair of slices is dedicated to receiving new voxels, while another two 
pairs of slices arc dedicated to supplying the next stages of the 
pipeline, i.e., the gradient estunation 220 and the voxel interpolation 
stages 240. 

In Figure 3, slices labeled ^ + 7 and s +2 are receiving new voxels, 
while slices j, 5-7, and s-2 are providing voxels to the subsequent 
stages. Slice s-S was used during a previous slice and is currently 
empty. When the processing of slice 5-2 is complete, slice 5-2 is also 
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marked as empty. Then, slices 5-2 and s-3 will become the new 
destination for voxel reads, while the processing modules will start 
taking their input from slices s +i, 5, and s-L 

The gradient estimation stage 220 estimates gradients at voxel points 
in slice s-1 from voxel values in slices s and 5-2. These gradients arc 
then stored in a two slice gradient buffer 225 for interpolation by unit 
228. Gradients for voxel points in slice 5-2 had been stored in the 
gradient buffer during a previous slice iteration. 

Next, voxel interpolation 240 and gradient interpolation 228 proceed 
concurrently. That is, a slice of samples is obtained by interpolating 
voxel values from voxel slices s -1 and s -2. Gradients at a sample 
points in the current slice of samples are obtained by interpolating the 
gradients of gradient slice s -1 and gradient slice s -2 in the gradient 
buffer. More than one slice of samples and interpolated gradients can 
be obtained from the same pair of voxel and gradient slices. 

The interpolated voxel values are next applied to the classification 
stage 230, which converts the values to interpolated RGBa color 
values. The RGBa values and the gradients are next supplied to the 
illumination stage 260, and finally to the compositing stage 270. 

In an optional mode, gradients arc extracted directly from, for 
example, three voxel fields, instead of using the gradient estimation 
stage 220 to estimating gradients from any voxel field 
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GCI - Gradient, Qassification, Inlerpolation 

Figure 4 aiustratcs the rendering algorithm for the GCI mode, also 
selected by signal 202. The voxel buffer 210 is filled as described 
above, and the gradient estimation process is also the same. The 
difference is in the voxel classification and interpolation side. In 
particular, raw voxels are taken from the voxel slice buffer, that is 
slice s -i, and immediately converted to RGBa values by the 
classification stage 230. These are then stored in a two slice RGBa 
buffer 235, Note, the RGBa buffer 235 is parallel to the two slice 
gradient buffer 225. 

The two slices are then input to the RGB a interpolation stage 240, 
where interpolated RGBa values are produced in parallel with 
interpolating gradients. These are then applied with interpolated 
gradients to the illumination and compositing stages to produce the 
pixel output 271 of the base plane image. These last two modules are 
identical to those of Figure 2-3. 

One additional optional mode is also possible. In this optional mode, 
multiple, for example, four, fields of the voxel are interpreted as raw 
RGBa values. In this case, the gradient may be estimated from the 
alpha (a) field. 

It should be noted that for some volume data sets and rendering 
modes it may be best to estimate gradient in a later stage of the 
pipeline, for example, after classification. 
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It diould be apparent that the multiplexers 251-253 can be replicated 
for other stages. Stages can selected and connected in a number of 
different orders to provide a rcconSgurablc rendering pipeline. Some 
stages can be de-selected to not process the volume data at all, for 
example, for some renderings gradient estimation and illumination 
may be skipped. 

Flexible Voxel Format 

As shown in Figure 5, a voxel 500 as used by the configurable 
rendering pipeline 200 includes a plurality of fields (V^, V„) 501- 
209, Each of the fields 501-209 can be specified as an offset and 
width in the voxel 500. Any of the fields can overlap as shown for 
fields Vj and Vj. The fields can be enumerated in any order. 

The fields describe different attributes of a represented three- 
dimensional object or model. For example, if the object is a human 
head, then the fields 501-209 can respectively store intensity values 
acquired from CT, MRI, PET, SPECT, and ultrasound scans, ix., 
each voxel may store five different scan intensity values in a single 
volume representation. Alternatively, the scans can be stored as 
multiple separate volumes wherein each voxel contributes one field. 

Some fields can be category fields related to the way the volume is 
segmented, e.g., manual, semi-automatic, or automatic segmentation. 
In medical applications, segmentation can categorize bone, tissue, 
etc., In physical applications, segmentation can identify sub- 
assemblies or other parts to be rendered in a particular way. For 
physical models, the fields can store stale variables used in scientific 
visualization, e.g., pressure, velocity, angular momentum, elasticity, 
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density, temperature, and viscosity. For any volume data set, the 
fields can also store RGB a values, depth, 3D stencil, shadows, fog, 
voxelized and embedded geometry volumes, or gradients. 

For a multi-field voxel 500 according to our invention, the user can 
specify which fields to use for gradient calculations. For each 
component of the gradient, we can specify which of the voxel fields 
to use for that component. 

For multi-field visualization, it is usually desirable to interpolate 
fields within voxels separately. Furthermore, each field within the 
voxel can have a different interpolation fonction applied, e.g., Iri- 
linear for intensity fields, and nearest neighbor interpolation for 
category fields. The flexible voxels as described herein enable a 
common framework for treating all special cases of voxel formats in 
a uniform fashion. 

Field Format Register 

Figure 6 shows a field format register 600 that enables multiple field 
voxels according to the invention. In one embodiment, fields of 
voxels are defined by descriptors of the field format register. This is 
an 8-bit (7:0) descriptor defining the size of the field (in 4-bit nibbles) 
610, the position of the field within its voxel 620, pixel (also in 4-bit 
nibbles), and what to do (control 630) when the field is a different 
size than specified for its intended use. 

The control bits define how a field may adapted to fit the data path of 
the pipeline through which the field will pass. The field can be 
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changed by repeating fraction arithmetic, or by adding or removing 
bits from cither the most significant or least significant end. 

Control = 0: the field of the raw voxel is treated as an unsigned 
repeating fraction representing a number in the range [0...1]. To 
expand or shrink the repeating fraction to fit the data path, repealing 
fractional arithmetic is applied to scale and round, thereby 
representing the number with fewer or more bits of precession. 
Repeating fraction number representation is described in greater 
detail below. 

Control = 1: the field of the voxel is treated as a signed repeating 
fraction in the range 

Control = 2: the field of the raw voxel is expanded or truncated in its 
least significant bits to fit the data path. The most significant bits are 
preserved* 

Control = 3: the field of the raw voxel is expanded or truncated in its 
most significant bits to fit the data path. The least significant bits are 
preserved. 

Repeating Fraction Number Representation 

Many graphics applications use a fixed width binary number to 
represent color, transparency, or other parameters that have values in 
the range zero to one, inclusive. 

Lcl /? be the number of bits in the binary number and let Vbc the 
unsigned binary value stored in these bits. Then F = V/ (2^ - 1) is 2i 
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rational number in the range [ft.i]. That is, when Vequals zero, F 
equals zero, and when V equals its largest possible value, (2* - 1\ F 
equals one. This representation is well known in the prior art. For 
example, the OpcnGL Specification refers to it as a special kind of 
fixed point representation. 

To clearly distinguish the representatkjn described herein from 
ordinary fixed point representation, the term "repeating fractions" is 
used. The name term derives from the fact that expanding F into a 
fixed point binary fraction produces ^ WWW..., that is, a binary 
fraction that repeals the R-h\X value V infinitely to the right of the 
binary point. 

Repealing fractions can be represented with more than/? bits and can 
even be signed. In that case, R is the "repeating precision" of the 
number, since R defines the hnplicit scale factor (2* - 7). This altows 
F to have values outside the range {0,.J\ In this case, the binary 
fixed point representation consists of an integer value followed by an 
R'biX infinitely repeating binary value. Repeating fractions with the 
same precision may be added and subtracted in the same way as 
ordinary integers. 

Other arithmetic operations, including changing the repeating 
precision, may be performed by first computing F for each repeating 
fractkin, performing normal arithmetic, and then multiplying the 
resulting value by (2^ - 1) for the repeating precision R of the result. 
More efficient forms exist for operations on repeating fractions. For 
example, doubling the repeating precision from R to 2R simply 
requires computing V + (V«R). 
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Voxel Formats 

The pipeline according lo the invention allows a wide range of input 
formats of voxels. Input voxels can be 8, 16, 32, or larger bit 
quantities. Fields can be 4, 8, 12, or 16 bits in width, and are aligned 
on a 4-bit boundary within the voxel. All voxels fields are scaled to 
fit their data paths, which are typically twelve bits wide. 

The format of a voxel is described in a voxel format register 700. An 
example of the format of a 32-bit voxel 800 is illustrated in Figure 8. 
In Figure 7, the format register for the example voxel, FieldO 
occupies bits 11:0, Fieldl occupies bits 23:20, Field! occupies bits 
31:24, Fields overlaps F/eWO and occupies bits 15:8. For the 
example shown in Figure 8, the descriptions of the fields in field 
format register are as follows: 



Field3: Size = 


1 


Position = 


2. 


FieldZ: Size = 


1 


Position = 


6. 


Fieldl: Size = 


0 


Position = 


S, and 


FieldO: Size = 


2 


Position - 


0. 



Advantages 

Having flexible format voxels, and a reconfigurable pipeline enable a 
number of advantages. For example, one of the fields in a voxel can 
be used for category bits. These are extra bits in voxels that identify 
the voxel as part of some particular tissue, sub-assembly, or other 
partition of the volume. They are not interpolated, but they do 
contribute to the assignment of RGB a values to voxels. When 
category bits are used, classification usually precedes interpolation. 



(e9))02-183747 ( P 2 0 0 2 - 1 8dJL8 



The flexible voxel format allows gradients to be estimated from any 
selected field of the voxel. In these cases, a convolution kernel, such 
as a central difference, is applied to the selected field of each voxel io 
obtain the x, y, and z-components of the gradient of that voxel. These 
gradients are then interpolated, in parallel with raw or classified 
voxels, in order to obtain gradients at sample points. These arc then 
applied, along with RGB a values to the illumination stage 260, and 
finally to the compositing stage 270. 

In some applications and for some volumes, it may be better to use 
gradients that arc determined in some other manner, instead of 
estimating them from the fields of voxels on the fly. Flexible format 
voxels accommodate this mode. In particular, gradients may be 
precomputed for the voxels with a high level of precisioa These 
precompulcd gradients can be stored in one of the fields of the voxels. 
These precomputed gradients can bypass the estimation stage, but are 
interpolated and applied in the iUumination stage 260 just as the 
gradients estimated on the fly. 

In a further variation, precomputed gradient are applied to a 
convolution kernel for gradient estimation. This has the effect of 
taking the second partial derivative of the volume data set. By the 
same technique, higher order partial derivatives may be obtained. 
Such derivatives are useful for extracting weak surfaces in the 
volume data set. As described for Figure 9 below, this can be done by 
passing the volume data set through the pipeline (1-3) muhiple times 
in a "muhi-pass" operation. Each pass processing the volume data set 
with different set of rendering parameters. 
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The pipeline 200 processing flexible format voxels also admits 
volume data sets that are preclassified and presented as "RGB (X " 
voxels. Two alternatives are useful. In one, gradient estimation and 
the classification and shading are be skipped entirely, and the voxels 
simply interpolated In the other alternative, gradients are estimated 
from alpha values or a luminance function, and shading the RGBa 
values is done according to the illumination function. Note, the 
pipeline 200 according to the invention is also capable of 
interpolating both voxel-gradient values and RGBa values. 

Multi-pass Valume Rendering 

The rendering pipeline 2(X3 can also write out a volume data set that 
has passed through some, but not necessarily all, of the stages of the 
pipeline 200 back to the voxel memory, see Figure 9. In an example 
use, the configured pipeline renders a set of voxels into RGBa values. 
Instead of composited along rays, the RGBa values are stored back 
to the memory 201 as RGBa voxels in a three-dimensional voxel 
array. Then, the volume is rerendered with a different set of rendering 
parameters. This process can repeat until a final volume is present in 
memory 201. The final accumulated values are then rendered one 
final time to generate an image in the pixel memory 209* 

This technique enables a number of features, such as fast resampling. 
With fast resampling, the volume data set can be resampled to a 
different resolution using the speed and power of the volume 
rendering pipeline and volume memory, instead of relying upon the 
host processor and software. 
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Multi-pass rendering can be used to produce complex shadows on the 
volume. One pass is needed for each light source. Each pass is 
accumulated in the output volume data set, and then a final pass 
interpolates the results and projects them onto the base plane. In 
multi-pass rendering, the volume data set output from a current pass 
is combined with an already existing volume data set in the memory* 
This sometimes requires a read-modify-write operation rather than a 
simply write operation. 

Multi-Channel Rendering 

The pipeline 200 can also process multi-channel data sets. Some 
scanning techniques, such as, ultrasound and seismic applications, 
have data of 

more than one type, each with its own classification- During 
rendering these data sets are be superimposed on each other, and 
combined, voxel-by-voxel 

More particularly, the compositing stage 270 is able to operate in 
either of two modes. In a first mode, the RGBa values are combined 
with previously stored pixel values, and in a second mode, a ray of 
RGBa values is accumulated, and the result is combined with a 
previously stored pixel value from some previous pass. 

Pipeline General Structure 

Generally, the relation of the pipeline 200 and the memory 201 can 
be as shown in Figure 9. H^e, the stages (stagey, stagCj, stage^) 1- 
3 of the pipeline 200 are connected to each other by multiplexers 4 so 
that for a particular rendering application, the stages are ordered by a 
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select signal 5. The input to the pipeline is a raw volume data set 
stored in a memory 6. The output of the pipeline is a modified 
volume data set. Individual data items of the volume data set passed 
between the pipeline and the memory are flexible format voxels 7. 
The volume data set can be processed by multiple passes. 

It is to be understood that various other adaptations and modifications 
may be made within the spirit and scope of the invention. Therefore, 
it is the object of the appended claims to cover all such variations and 
modifications as come within the true spirit and scope of the 
invention. 

4 Brief Descripton of Drawings 

Figure 1 is a block diagram of a prior art rendering pipeline; 
Figure 2 is a block diagram of a configurable rendering pipeline; 
Figure 3 is a pipeline with interpolation before classification; 
Figure 4 is a pipeline with classification before interpolation; 
Figure 5 is a block diagram of a flexible format voxel. 
Figure 6 is a block diagram of a field format register; 
Figure 7 is a block diagram of a voxel format register; 
Figure 8 is an example formatted voxel; 

Figure 9 is a block diagram of a configurable rendering pipeline 
connected to a volume memory. 
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1 Abstract 

A volume rendering pipeline includes a plurality of 
processing stages such as a gradient estimation stage, an interpolation 
stage, a classification stage, an illumination stage, and a compositing 
stage. The stages are connected to each other by multiplexers. A first 
multiplexer connects an output of a first stage to an input of a second 
stage. A second multiplexer connects an output of the second stage to 
an input of the first stage. A third multiplexer has inputs connected to 
the output of the first stage and the output of the second stage, the 
first, second, and third multiplexers are responsive to a select signal 
to configure the stages of the rendering pipeline for processing the 
volume data set. 

2 Representative Drawing 

FlG.l 
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